More information on database performance with FME comes in the later lesson, Optimize Database Performance.
After completing this lesson, you’ll be able to:
There are various ways to speed up writing data. Compared to reading, tuning the underlying systems is a more important improvement; the number of features is less important as it's much harder to write extra data unintentionally.
If you are writing to a file system then make sure the disk is fast and responsive – use solid-state drives – and that the operating system is not busy writing other files at the same time, as the latter could cause a significant bottleneck.
Also, check if you are using RAID; some configurations need multiple writes and can slow down a translation.
If you are writing to a database, then existing indexes and joins can cause the process to be slower than expected. In many cases, it will be quicker to drop an index, write the data, and then recreate the index.
Perhaps the most important technique for improving writer performance involves the scenario where a workspace has multiple writers. In short, it’s important to get the writers into the optimum order, to ensure that the writer that is to receive the largest amount of data is written first.
The reasoning behind this is that the first writer in a workspace starts to write data as soon as it is received. Other writers cache theirs until they are ready to start writing. Therefore, if the largest amount of data is written immediately, lesser amounts of data have to be written to, and stored in, a cache. This can improve performance tremendously, particularly when the translation is especially unbalanced; for example, one million features go to one writer, and only ten features go to another.
There are two ways to affect the order that writing occurs.
Firstly each writer is listed in the Navigator window in Workbench and can be re-ordered by moving them up and down in the list in the Navigator window:
The first writer in the list is the one that is initiated first, therefore it should be the one to receive the most data.
The second method is to use a workspace parameter called Order Writers By:
This parameter can be left to Position in Workbench Navigator in which case the order of writers as defined in the Navigator takes priority. Alternatively, it can be set to First Feature Written. In that case, the writer that receives the first feature will be the first to start writing data.
If you saw this workspace, would you know how to order the writers in the Navigator? Answer in the Quiz below.
